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Telemedicine  Satisfaction  Among  Naval  Radiologists 


Summary 


Background 

This  study  aimed  to  determine  provider  satisfaction  with  teleradiology  in  a  military  hospital. 
This  study  used  a  survey  to  assess  radiology  specific  technologies  not  addressed  in  previous 
research,  such  as  wireless  radiography  and  computed  radiographs  (CR).  This  research  also 
describes  the  perception  and  use  of  telemedicine  by  providers  in  two  different  medical 
specialties  —  otolaryngology  and  radiology  —  to  determine  the  feasibility  of  using  the  same 
survey  instrument  to  measure  satisfaction  in  different  telemedicine  specialties  (Whitten  & 
Mair,  2000).  Similar  survey  data  from  an  earlier  evaluation  of  ear,  nose,  and  throat  (ENT) 
specialist  physicians  and  nonphysician  medical  personnel  at  the  Naval  Medical  Center,  San 
Diego  (NMCSD)  were  included  in  the  present  paper  for  this  purpose. 


Method 


A  60-item  survey  assessing  satisfaction  with  telemedicine  technologies  was  administered  to 
20  radiologists  at  NMCSD.  The  surveys  were  distributed  during  a  staff  meeting  where 
respondents  completed  them  in  15  minutes  and  did  not  interact  with  the  researcher.  Data 
from  a  previous  study  of  mostly  ENT  providers  and  their  cases  are  included  in  the  present 
study.  The  ENT  data  were  collected  by  a  comparable  survey  administered  to  3  ENT  medical 
specialists,  5  physicians,  and  7  nonphysicians  via  telephone  interviews  between  November  1, 
2000  and  March  1 , 2001 .  Questions  assessing  the  general  perceptions  of  telemedicine  were 
identical  for  both  studies.  The  differences  between  the  two  surveys  are  primarily  in  the  details 
of  technologies  listed  in  particular  to  each  specialty.  Items  measuring  satisfaction,  perception, 
and  usefulness  were  presented  on  a  5-point  Likert  scale.  The  history  of  use  and  demographic 
data  were  presented  in  closed-ended  categorical  scales  or  numerical  scales.  A  final  open- 
ended  question  was  included  to  capture  other  possible  answers  not  included  in  the  scales. 


Results 


Radiology  personnel  reported  generally  positive  attitudes  toward  telemedicine  technologies 
and  the  telemedicine  process  at  NMCSD.  Results  were  similar  to  those  obtained  during  the 
previous  study  from  ENT  medical  personnel,  who  also  reported  generally  positive  attitudes  on 
telemedicine  technologies  and  the  telemedicine  process. 

Ninety  percent  (90%)  of  the  radiologists  reported  having  used  CR  in  their  jobs  and  they  rated 
it  high  in  usefulness  (4.1  on  a  5-point  scale).  Residents  reported  1.7  years  of  use  and  range 
of  1  to  3  years,  while  radiologists  reported  4.2  years  of  use  and  range  of  1  to  10  years.  These 
radiologists  rated  wireless  technology  a  3.8  in  usefulness,  also  on  the  positive  end  of  the 
scale.  Of  the  55%  of  radiologists  reported  to  have  used  wireless  radiography,  residents 
reported  an  average  1 .7  years  of  use  and  a  range  from  1  to  2  years.  Radiologists  reported  an 
average  of  4.7  years  of  use  with  a  range  from  2  to  10  years. 

Conclusions 

Respondents  in  both  radiology  and  ENT  specialties  were  generally  satisfied  with  the 
telemedicine  technology  and  reported  similar  positive  attitudes  about  telemedicine  and  its 
role  in  health  care  in  general.  The  ENT  medical  personnel  responded  positively  to 
telemedicine  technologies  used  in  their  practice  (e.g.,  Referral  Management  System  [RMS] 
and  video  teleconferencing  [VTC]),  and  the  radiology  personnel  responded  positively  to 
specific  radiological  technologies  (e.g.,  wireless  and  CR). 
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These  results  suggest  that  the  present  survey  instrument  may  be  useful  in  providing  data  for 
systematic  comparisons  of  these  two  telemedicine  specialties  and  possibly  other  widely  used 
specialties  such  as  dermatology. 


Telemedicine  Satisfaction  Among  Naval  Radiologists 


Introduction 


Teleradiology  involves  the  transmission  of  medical  images,  such  as  x-rays  or  magnetic 
resonance  imaging,  to  a  distant  radiologist  for  interpretation  (Linkous,  2002).  Teleradiology 
enables  a  provider  to  have  more  immediate  access  to  a  radiologist/radiology  specialist  for 
consultation.  Radiology  is  the  most  mature  and  rapidly  evolving  specialty  in  telemedicine 
(Abdullah,  Ng,  &  Pathmanathan,  1999;  Hailey,  Roine,  &  Ohinimaa,  2002).  Although 
diagnostic  accuracy  does  not  necessarily  improve  with  teleradiology,  accuracy  does  not 
diminish  as  compared  with  conventional,  film-based  radiology  (Franken,  1998).  Telehealth 
improves  performance  standards  in  the  delivery  of  health  care  by  creating  information-rich 
learning  environments  and  bringing  best  practices  from  around  the  world  to  carriers  and 
practitioners  wherever  they  are  located  (Australian  and  New  Zealand  Telehealth  Committee, 
1996).  Therefore,  this  technology  allows  clinics  without  radiologists  to  access  one  and  makes 
travel  for  patients  and  radiologists  unnecessary. 

High  physician  and  patient  satisfaction  with  telemedicine  has  been  documented  in  several 
medical  specialties,  including  dermatology  (Lowitt,  Kessler,  Kauffman,  Hooper,  Siegel,  & 
Burnett,  1998),  telepsychiatry  (Hilty,  Luo,  Morache,  Marcelo,  &  Nesbitt,  2002;  Simpson,  Doze, 
Urness,  Hailey,  &  Jacobs,  2001),  vascular  surgery  (Endean,  Mallon,  Minion,  Kwolek,  & 
Schwarcz,  2001),  pediatric  cardiology  (Mehta,  Wakefield,  Kienzle,  &  Scholz,  2001),  and  tele¬ 
ultrasound  (Smith  &  Brebner,  2002).  Lowitt  and  colleagues  (1998)  also  found  that  the 
telemedicine  consultation  process  improved  the  patient-physician  bond,  the  attending 
physician’s  level  of  confidence,  and  in-person  care. 

Studies  on  CR  showed  radiologist  satisfaction  to  be  high  after  an  initial  warm-up  period.  Willis 
and  Parker  (1999)  found  that  it  takes  about  3  months  for  experienced  radiologists  to  reset 
their  standards  of  “normal”  to  accommodate  the  change  in  contrast  for  the  new  CR  images, 
(which  generally  have  more  contrast  than  conventional  images.)  However,  most  of  the 
studies  in  teleradiology  center  on  the  resolution  requirements  for  digital  teleradiology  and 
comparisons  of  diagnostic  performance  with  traditional  radiology.  Therefore,  questions 
remain  regarding  actual  service  performance  using  teleradiology,  particularly  in  a  military 
setting  where  CR  images  are  increasingly  used  by  remote  treatment  sites  such  as  shipboard 
medical  departments. 

The  U.S.  military  has  been  an  effective  supporter  of  digital  imaging  and  teleradiology  for  the 
past  19  years.  Beginning  as  a  timesaving  tool  for  the  Naval  Academy,  this  technology  grew 
into  a  fully  functional  radiology  support  network  for  deployed  ships,  used  primarily  as  tools  to 
transmit  x-ray  images  from  ship  to  shore.  Images  are  transmitted  from  ships  to  large  medical 
facilities  such  as  NMCSD,  the  National  Naval  Medical  Center  in  Bethesda,  MD,  and  hospitals 
in  Portsmouth,  VA,  where  staff  radiologists  can  examine  them  and  send  back  diagnoses.  The 
digital  imaging  and  teleradiology  technology  falls  into  two  categories:  machines  that  convert 
x-ray  images  into  digital  ones,  and  software  used  to  transmit  those  digital  images.  CR,  the 
method  of  converting  x-ray  images  to  digital  ones,  is  being  used  increasingly  on  large  ships 
and  has  numerous  advantages  over  film-based  methods  (Karpf,  2001).  Radiographic  images 
are  exposed  onto  phosphor  plates  and  then  converted  directly  to  digital  images,  eliminating 
the  need  to  develop  film.  This  also  eliminates  problems  resulting  from  the  shipboard  handling 
and  disposing  of  chemicals  used  in  the  developing  process.  Brumage,  Chinn,  and  Cho  (2001) 
found  teleradiology  provided  military  hospitals  significant  cost  savings  over  a  1-year  period. 
However,  there  have  been  no  studies  thus  far  on  provider  satisfaction  with  teleradiology  in  a 
military  setting.  This  research  responds  to  a  general  directive  to  determine  the  factors  that 
influence  telemedicine  impact  and  use,  and  provides  teleradiology  satisfaction  data. 

Telemedicine  attitudes  (e.g.,  perception  of  usefulness)  and  experience  are  positively 
associated  with  telemedicine  use  (Karp  et  al.,  2000).  A  decline  in  physician  satisfaction  could 
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lead  to  lower  retention  rates  for  military  physicians,  as  well  as  lower  levels  of  physician 
performance,  patient  satisfaction,  and  quality  of  care  (Rattelman,  1996).  Teleradiology  serves 
ship  populations  (Stoloff,  Garcia,  Thomason,  &  Shia,  1998),  smaller  facilities,  and  the  Navy  in 
general,  by  helping  to  divide  the  workload  more  evenly  among  all  Navy  radiologists. 
Telemedicine  helps  radiologists  at  remote  stations  by  reducing  their  continually  intense 
workload  (Mun  et  al.,  1998).  Using  telemedicine  to  balance  the  workload  helps  with  retention 
by  ensuring  that  no  one  is  overworked,  and  it  addresses  the  issue  of  physician  shortages  by 
allowing  Navy  medicine  to  make  full  use  of  available  resources  (Karpf,  2001). 

For  this  study,  access  was  provided  to  the  NMCSD  radiology  department  that  uses 
teleradiology  for  ship-to-shore  electronic  transmissions  of  CR  images  in  support  of 
nonradiological  shipboard  providers.  Teleradiology  use  affects  patient  care,  including 
diagnosis  and  treatment,  in  similar  nonmilitary  contexts  where  radiologists  reviewed  the  initial 
readings  of  radiographs  by  nonradiologists  (e.g.,  Kiuru,  Paqakkala,  Kallio,  Aalto,  &  Rajamaki, 
2002).  The  present  study  included  careful  assessment  of  provider  satisfaction  with  radiology- 
specific  technologies  not  addressed  in  previous  studies,  such  as  wireless  radiography  and 
CR.  Based  on  the  literature  (Karp  et  al.,  2000;  Hunter  et  al.,  1999;  Lee  et  al.,  1998;  Lowitt  et 
al.,  1998),  a  high  satisfaction  rate  among  military  radiologists  was  expected. 

The  present  study  also  included  data  from  a  prospective  study  of  the  ENT  specialty  that 
assessed  attitudes  and  experiences  with  TRICARE  Region  9  telemedicine  (Melcer, 

Hunsaker,  Crann,  Caola  &  Deniston,  2002).  In  the  future,  such  comparisons  could  assist  in 
assessing  the  general  provider  satisfaction  across  different  telemedicine  specialties. 
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Method 


The  methodology  for  this  study  was  a  self-administered  survey.  This  study  was  a  single-point 
assessment  of  use,  attitudes,  and  satisfaction  with  telemedicine  by  radiologists  at  NMCSD. 

Data  Sources  and  Subjects 

Observations  came  from  a  survey  administered  at  the  beginning  of  an  all-staff  meeting  for 
radiology  personnel  at  NMCSD.  Participation  was  voluntary  and  all  personal  identifying 
information  (e.g.,  name,  social  security  number)  was  excluded  from  the  data  collection. 

Medical  Personnel 

A  convenience  sample  of  radiology  personnel  from  NMCSD  completed  the  surveys.  These 
were  general  radiologists  (N  =  9)  and  resident  radiologists  (N  =  1 1 ),  1 8  males  and  2  females. 
All  but  one  of  these  twenty  individuals  were  active  duty;  one  resident  radiologist  was  in 
military  reserve.  Ten  percent  (N  =  2)  of  this  sample  had  been  ship  deployed  within  the  last 
year.  The  present  sample  of  twenty  was  less  than  half  (43%)  of  the  46  radiologists  at 
NMCSD.  The  46  radiologists  and  residents  at  NMCSD  include  17  active-duty  radiologists,  1 
full-time  civilian  radiologist,  26  residents,  and  2  active-duty  nuclear  medicine  specialists. 
Currently,  approximately  92  radiologists  serve  in  the  Navy. 

Naval  Medical  Center,  San  Diego 

NMCSD  is  part  of  TRICARE  Region  9,  which  provides  primary  health  care  for  active-duty  and 
retired  service  personnel,  their  eligible  family  members,  and  their  survivors.  The  radiology 
department  chiefly  uses  digitally  stored  radiographs  instead  of  film.  The  department  is 
currently  studying  CRs  taken  shipboard  to  determine  the  feasibility  of  using  CRs  to  facilitate 
storage  and  transmission  of  shipboard  and  land-based  consultations. 

Data  Analysis 

The  data  analysis  was  designed  to  address  the  following  questions: 

•  What  is  the  experience  of  radiology  and  ENT  personnel  regarding  telemedicine? 

•  Do  radiology  and  ENT  personnel  perceive  telemedicine  to  be  useful? 

Preliminary  trends  are  described  for  the  data  because  of  the  small  samples  of  9  general 
radiologists  and  1 1  resident  radiological  personnel.  The  ENT  data  from  Melcer  et  al.  (2002) 
contained  similar  sample  sizes.  Thus,  any  comparisons  between  ENT  physicians  and 
radiologists  should  be  interpreted  with  caution. 
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Results 


Medical  personnel  in  the  radiology  department  reported  generally  positive  attitudes  on 
telemedicine  technologies,  types  of  use,  quality  of  technologies,  and  the  telemedicine 
process  at  NMCSD.  Compared  with  resident  radiologists,  general  radiologists  had  used 
telemedicine  longer,  though  there  were  no  apparent  differences  between  these  two  groups  in 
types  of  telemedicine  use  or  frequency. 

Compared  with  ENT  participants  from  the  previous  study,  radiologists  in  the  present  study 
used  telemedicine  more  for  patient  education  (28%  of  radiologists  vs.  1 1 .5%  of  ENT 
personnel),  provider  education  (28%  of  radiologists  vs.  5%  of  ENT  personnel),  and  technical 
support  (43%  of  radiologists  vs.  5%  of  ENT  personnel).  Only  in  confirming  diagnoses  did  ENT 
personnel  score  higher  than  radiologists,  with  78%  using  telemedicine  to  confirm  diagnoses, 
versus  55%  of  radiologists. 

Demographics 

The  radiology  and  ENT  samples  consisted  predominantly  of  active-duty  males.  However,  the 
samples  did  include  females,  representing  approximately  20%  of  the  total  sample,  and 
civilians,  representing  around  11%  of  the  total  sample.  Fewer  than  5%  of  the  total  sample 
had  been  ship  deployed  within  a  year  prior.  The  radiologist  population  included  11  resident 
radiologists  and  9  practicing  radiologists  (see  Table  1).  The  group  in  the  previous  study  is 
given  the  ENT  designation  based  on  the  fact  that  such  consultations,  which  were  the  most 
common  for  this  region,  rely  on  real-time  VTC  (Melcer  et  al.,  2002).  Teleradiology,  on  the 
other  hand,  does  not  usually  rely  on  real-time  consultations  but  uses  a  store-and-forward 
mode  in  most  cases. 
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Table  1.  Radiologist  and  ENT  Demographics 


Radiologist 

Demographics 

ENT 

Demographics 

Total 

N 

% 

N 

% 

20 

100 

15 

100 

Gender 

Male 

17 

85 

11 

73 

Female 

3 

15 

4 

26 

Military  status 

Active  duty 

19 

95 

12 

80 

Civilian 

1 

5 

3 

20 

Ship  deployed  in  the  last  year 


Yes 

1 

5 

NA 

No 

19 

95 

NA 

Located  at  other  facility  in  the  last  year 


Yes 

5 

25 

NA 

No 

15 

75 

NA 

Title 


Radiologist 

8 

40 

NA 

NA 

Specialist 

1 

(nuc.  med) 

5 

3 

(neurology, 
child  psych, 
ENT) 

20 

Resident 

11 

55 

NA 

NA 

Physician 

NA 

5 

33 

Non-physician 

NA 

NA 

7 

(1  IDC,  4  med 
techs, 

1  nurse,  1  PA) 

46 

Note:  IDC  =  independent  duty  corpsman;  PA  =  physician’s  assistant 


Technology  Experience 

The  radiologists  in  this  study  and  the  ENT  personnel  in  the  previous  study  had  used  the 
telephone  to  confirm  a  diagnosis  and  had  access  to  and  used  e-mail  and  the  Composite 
Health  Care  System  (CHCS).  All  radiologists  and  86%  of  ENT  personnel  reported  using  the 
Internet  before,  while  88%  of  radiologists  and  only  55%  of  ENT  personnel  had  previously 
used  VTC.  Eighty-six  percent  of  ENTs  had  used  an  RMS. 

Ninety  percent  of  the  radiologists  had  used  CR  in  their  jobs,  and  they  rated  it  high  in 
usefulness  (4.1  on  a  5-point  scale).  Residents  reported  1 .7  years  of  use  and  range  of  1  to  3 
years.  Radiologist  reported  4.2  years  of  use  and  range  of  1  to  10  years.  Of  the  55%  of 
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radiologists  who  reported  past  use  of  wireless  radiography,  residents  had  used  this 
technology  for  1 .7  years  and  radiologists  for  4.7  years. 

Past  Telemedicine  Use 

Table  2  shows  the  medical  reasons  for  which  the  radiology  and  ENT  personnel  reported 
using  telemedicine  technologies.  Confirming  diagnosis  was  the  primary  reason  reported  for 
past  telemedicine.  All  of  the  18  radiology  personnel  who  responded  to  this  section  had  at 
some  point  used  at  least  one  of  the  telemedicine  technologies  to  confirm  a  diagnosis;  100% 
had  used  the  telephone  to  confirm  a  diagnosis.  The  ENT  personnel,  however,  were  more 
likely  to  use  all  the  telemedicine  technologies  for  diagnosis  confirmation. 

The  ENT  personnel  were  also  consistently  more  likely  than  all  the  radiology  staff  to  use 
telemedicine  for  patient  education.  The  radiologists  were  less  likely  than  ENT  personnel  to 
use  telemedicine  for  continuing  medical  education  (not  shown  in  Table  2).  An  average  of  28% 
of  the  radiologists  and  57%  of  the  ENT  personnel  used  telemedicine  for  continuing  medical 
education. 

Table  2.  Past  Use  of  Telemedicine  Technologies  by  Medical  Personnel 


Radiologists  N  =  20 

ENT  Personnel  N  =  15 

Telemedicine 

Technology 

Diagnosis 
Confirmation  % 

Patient 
Education  % 

Diagnosis 
Confirmation  % 

Patient 

Education 

% 

Telephone 

100 

44 

100 

58 

Fax 

70 

21 

88 

26 

E-mail 

79 

43 

100 

67 

Internet 

41 

62 

100 

80 

CR 

75 

NA 

NA 

NA 

Wireless 

34 

0 

NA 

NA 

VTC 

6 

0 

100 

54 

Overall  average 

55 

28 

96 

57 

Current  Access  to  Telemedicine  Technologies 

Table  3  shows  that  all  the  resident  and  general  radiologists  at  NMCSD  had  current  access  to 
various  telemedicine  technologies.  This  table  is  broken  down  between  residents  and  general 
radiologists  due  to  the  substantial  differences  of  years  worked  in  radiology.  All  personnel 
reported  well  over  1  year  of  experience  with  each  technology.  General  and  resident 
radiologists  reported  similar  technology  experience  but  reported  apparent  differences  in  VTC, 
wireless,  and  RMS.  This  difference  might  be  expected  because  the  residents  had  fewer  total 
years  of  medical  experience.  However,  they  still  had  a  substantial  amount  of  technology 
experience  and  a  reasonable  amount  of  telemedicine  experience.  On  average,  the 
radiologists  reported  more  years  of  use  in  each  technology  than  the  ENT  personnel  in  the 
previous  study. 


13 


Table  3.  Current  Access  to  Telemedicine  Technologies  by  Medical  Personnel 


General  Radiologists 
A/  =  9 

Resident 

Radiologists 

N  =  11 

ENT  Personnel 

N  =  15 

Technology 

Current 
Access  % 

Years 
Used  (M) 

Current 
Access  % 

Years 
Used  (M) 

Current 
Access  % 

Years 
Used  (M) 

E-mail 

100 

10.13 

100 

7.5 

86 

4.5 

RMS 

NA 

NA 

NA 

NA 

86 

1.6 

CHCS 

100 

8.63 

100 

8.25 

NA 

NA 

CR 

88 

4.2 

100 

1.7 

NA 

NA 

Wireless 

63 

4.7 

100 

1.7 

NA 

NA 

Internet 

78 

8 

75 

4.2 

86 

3.4 

VTC 

75 

3.17 

25 

5 

86 

1.7 

Note:  CHCS  =  Composite  Health  Care  System 


Purposes  of  Telemedicine  Use 

Table  4  shows  the  reasons  for  recent  telemedicine  use  by  radiologists  (over  the  last  4 
months)  and  by  ENT  personnel  during  a  similar  period  in  the  previous  study.  Differences 
between  the  radiologists  and  ENT  personnel  were  visible  in  the  overall  distribution  of 
telemedicine  purposes.  The  radiologists  were  considerably  more  likely  than  the  ENT 
personnel  to  use  telemedicine  for  technical  support.  In  contrast,  the  ENT  personnel  were 
more  likely  to  use  telemedicine  to  confirm  diagnoses.  These  results  should  be  interpreted 
with  caution  due  to  the  general  radiologists  high  percentage  of  unanswered  usage  questions 
(25.7%). 
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Table  4.  Recent  Telemedicine  Usage  by  Medical  Purpose 


Confirm 

Diagnosis 

% 

Patient 

Education 

% 

Continue 

Medical 

Education 

% 

Technical 

Support 

% 

Unknown 

% 

Radiologist 
(N  =  9) 

28 

9 

7.1 

30.2 

25.7 

Resident 
Radiologist 
(N  =  11) 

54 

6 

20 

20 

0 

ENT  Physician 
(N  =  8) 

68 

3 

9 

9 

11 

ENT 

Non-physician 
(N  =  7) 

77 

16 

1 

1 

5 

Level  of  Telemedicine  Activity 

Table  5  summarizes  radiological  personnel  reports  of  overall  patient  load  and  telemedicine 
activity  during  the  single-point  study  period.  The  telemedicine  ratio  for  radiologists  is  very  low 
in  this  table,  and  the  actual  ratio  may  be  higher.  The  radiologists  may  not  have  perceived  the 
computed  images  they  worked  on  as  telemedicine  patients.  For  future  studies,  it  may  be 
worth  clarifying  this  question  so  that  respondents  treat  CR  images  as  telemedicine  cases. 

Table  5  also  reveals  that  ENT  personnel  in  the  previous  study  reported  substantially  more 
telemedicine  activity  than  radiologists  in  the  present  study.  This  is  seen  in  the  number  of 
telemedicine  cases  and  in  the  ratio  of  telemedicine  cases  to  all  patient  care  during  the  study 
period  (telemedicine  cases/telemedicine  and  nontelemedicine  cases).  Note  that  these  are 
discrete  means  based  on  all  respondents,  and  the  ratio  is  not  a  ratio  of  means. 


Table  5.  Telemedicine  Activity  Reported  by  Medical  Personnel 


Telemedicine  Activity  (last  4  months) 

Radiologists 
N  =  20  (M) 

ENT 

N='\5(M) 

Telemedicine  patients  seen  (TMED) 

37 

124 

Total  patients  seen  (TMED  +  non-TMED) 

1936 

1159 

Telemedicine  ratio  (TMED/TMED  +  non-TMED) 

5% 

37% 

Satisfaction  With  Telemedicine 

Figure  1  shows  mean  scores  (+/- 14  standard  deviation)  for  medical  personnel  attitudes 
toward  satisfaction  with  technologies  (e.g.,  e-mail,  VTC),  satisfaction  with  different  modalities 
(images,  sound,  written  text)  transmitted  by  telecommunications,  usefulness  of  technologies, 
and  overall  perception  of  telemedicine  for  health  care. 

Figure  1  indicates  that  ENT  personnel  in  the  previous  study  reported  consistently  favorable 
attitudes  toward  telemedicine  across  all  types  of  questions.  Radiologists  in  the  present  study 
were  slightly  lower,  though  still  in  the  “satisfied”  area  of  the  scale.  They  rated  their  attitudes 
on  a  5-point  scale,  with  more  positive  attitudes  indicated  by  higher  scores  (e.g.,  5  =  very 
satisfied,  4  =  somewhat  satisfied,  3  =  undecided,  2  =  somewhat  dissatisfied,  1  =  very 
dissatisfied).  Similar  scores  were  reported  for  radiologists  and  ENT  personnel  averages 
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within  each  of  the  four  attitude  variables.  Each  of  the  four  attitude  variables  consisted  of  six  to 
ten  different  questions. 


Figure  1.  Telemedicine  Attitude  Scores  Reported  by  Medical  Personnel 

Usefulness  received  the  highest  ratings  of  the  four  attitude  variables  for  radiologists,  with  CR 
scoring  the  highest  and  Internet  second.  Responding  to  questions  about  their  satisfaction 
with  specific  modes  of  transmitted  information,  the  radiologists  rated  phone  and  e-mail  the 
highest.  Since  RMS  and  store-and-forward  were  not  applicable  to  many  radiologists,  many 
did  not  answer  or  replied  with  an  “undecided”  score  of  3.  ENT  personnel  gave  high  ratings  on 
the  usefulness  of  VTC  (M  =  4.45).  In  order  to  ensure  there  was  no  response  set  in  the  survey 
(i.e.,  agreement  with  all  statements),  three  questions  were  worded  negatively  so  that 
respondents  would  have  to  disagree  to  indicate  a  favorable  attitude  toward  telemedicine: 

•  The  quality  of  patient  care  rendered  through  the  use  of  telemedicine  is  inferior  to  that 
provided  in  person. 

•  Existing  telemedicine  systems  do  not  meet  my  practical  needs  as  a  healthcare 
provider. 

•  Most  existing  telemedicine  systems  are  too  limited  to  be  effective. 

The  average  rating  for  these  three  questions  was  2.8,  which  was  on  the  negative  end  of  the 
scale. 

The  attitude  portion  of  the  survey  consisted  of  different  sections  designed  to  measure  three 
separate  constructs:  satisfaction,  usefulness,  and  perception  of  telemedicine  and  health  care. 
Internal  consistency  of  this  survey  is  important  to  determine  if  the  instrument  will  perform  the 
same  when  administered  at  different  times  to  the  same  sample.  Therefore,  the  questions 
intended  to  measure  each  construct  were  evaluated  for  reliability  using  Cronbach’s  alpha. 
Questions  27-36  on  the  survey  regarding  satisfaction  in  general  were  combined  with 
questions  37-42  covering  satisfaction  of  particular  technologies  to  calculate  the  reliability.  The 
following  standardized  item  alphas  resulted:  satisfaction,  a  =  .75;  usefulness,  a  =  .93; 
perception  of  telemedicine  and  health  care,  a  =  .74.  Reliability  scores  are  considered 
acceptable  at  .70  (Nunnally,  1978),  therefore  these  scores  are  considered  high,  especially  for 
usefulness. 

The  same  attitude  portion  of  the  survey  for  the  previous  ENT  survey  was  also  high,  with 
reliability  scores  of  satisfaction,  a  =  .87,  and  perception  of  telemedicine  and  health  care,  a  = 
.94.  Usefulness  scored  at  a  =  .64.  This  score  is  considered  low  and  could  be  due  to  the  data 
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for  “usefulness”  being  multidimensional,  rather  than  because  of  poor  reliability.  When  data 
have  a  multidimensional  structure,  Cronbach’s  alpha  will  usually  be  low.  A  check  of 
multidimensionality  revealed  there  were  at  least  two,  possibly  three  subsets,  though  a 
reliability  analysis  of  these  subsets  was  not  possible  due  to  the  small  sample  sizes  in  the 
present  research. 

Figure  2  below  indicates  that  the  general  and  resident  radiologists  in  the  present  study 
reported  favorable  attitudes  on  teleradiology  technologies.  The  average  scores  (+/-  V. 2 
standard  deviation)  were  calculated  for  satisfaction  in  each  area.  The  two  groups  showed 
similar  mean  scores  on  satisfaction  with  modality  quality,  usefulness  of  technologies,  and 
perception  of  teleradiology-specific  technologies. 


Radiography  Attitude  Scores  Reported  by  Radiologists 


Satisfaction  Satisfaction  Satisfaction  Satisfaction  Usefulness  of  Usefulness  of 
with  CR  with  wireless  with  CR  with  wireless  CR  wireless 

quality  quality 


Figure  2.  Radiography  Attitude  Scores  Reported  by  Radiologists 

Comments  by  Medical  Personnel 

At  the  end  of  the  structured  question  section  of  the  surveys  in  the  previous  and  present 
studies,  the  medical  personnel  were  asked  if  they  had  additional  comments.  The  radiology 
personnel  offered  no  comments.  ENT  personnel  supplied  favorable  comments  about  the 
RMS  and  VTC,  but  reported  unacceptably  low  quality  for  child  psychiatry  (Melcer  et  al., 
2002).  Other  comments  indicated  that  other  types  of  telemedicine,  including  video,  would  be 
useful.  The  ENT  data  were  gathered  via  telephone,  which  may  have  elicited  more  comments 
than  the  self-administered  radiology  survey. 
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Discussion 


The  present  results  support  several  conclusions: 

•  The  radiology  and  ENT  personnel  at  NMCSD  had  substantial  telemedicine 
experience,  and  both  groups  of  providers  reported  positive  attitudes  toward  the 
telecommunications  technologies  they  used,  and  the  role  of  telemedicine  in  health 
care  processes  in  general. 

•  Most  previous  military  radiology  studies  focus  on  patient  satisfaction  or  physician 
agreement  between  teleradiology  and  film.  The  present  study  demonstrates 
favorable  attitudes  on  teleradiology  in  the  present  sample  of  radiologists.  The  present 
survey  revealed  a  trend  for  relatively  low  satisfaction  with  wireless  versus  CR,  and 
therefore  the  survey  may  be  useful  for  evaluating  satisfaction  with  alternative 
radiology  technologies  in  large-scale  studies. 

•  Radiologists  rated  teleradiology-specific  technologies,  such  as  CR  images  and 
wireless,  high  in  satisfaction.  However,  a  trend  seemed  to  emerge  suggesting  greater 
satisfaction  for  CR  than  for  wireless  and  CR.  One  of  the  factors  that  could  be 
contributing  to  this  pattern  of  preference  is  the  advantage  in  image  modification  of  CR 
over  wireless.  Pathi  and  Langlois  (2002)  found  that  digital  radiography  offered 
significant  image  modification  advantages  over  wireless,  meaning  that  the  digital 
images  could  be  manipulated  to  aid  viewing.  Another  factor  that  could  be  contributing 
to  the  pattern  of  preference  is  the  existence  of  problems  in  the  speed  and 
connectivity  of  wireless  transmission  (Yamamoto  &  Williams,  2000). 

•  The  success  in  using  the  satisfaction  survey  instrument  across  two  telemedicine 
disciplines,  ENT  and  radiology,  indicates  that  the  instrument  may  be  useful  in 
systematic  comparison  of  multiple  telemedicine  specialties. 
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Attitudes  and  Telemedicine  Experience 

Radiologists  in  this  study  reported  that  teleradiology  is  useful  with  positive  experiences.  A 
trend  worth  investigating  was  the  possible  relationship  between  the  experience  and  attitudes 
of  the  radiology  personnel:  general  radiologists  reported  more  years  of  experience  with 
technology  than  the  residents  and  also  slightly  higher  overall  attitude  scores  for  CR  (see 
Table  3  and  Figure  2).  Telemedicine  attitudes  do  correlate  positively  with  telemedicine 
experience  in  at  least  one  study  (Karp  et  al.,  2000),  and  the  radiologists  in  general  gave 
accounts  of  possessing  significant  computer  and  technological  experience.  However, 
significant  correlations  were  not  found  between  telemedicine  experience  and  satisfaction  in 
the  radiologist’s  responses.  Also,  ENT  personnel  in  the  previous  study  rated  telemedicine 
slightly  higher  on  the  whole  than  radiologists,  although  they  had  fewer  years  of  overall 
experience  with  telemedicine  than  radiologists.  Multivariate  analyses  of  variance  were 
performed  to  assess  differences  in  attitude  scores  between  radiologists  and  ENTs  in  Figure  1 
and  between  radiologists  and  residents  in  Figure  2.  The  analyses  suggest  no  significant 
differences  in  attitudes  across  groups  (multivariate  p's  >  .10.) 

Recommendations  and  Future  Work 

Several  recommendations  have  developed  as  a  result  of  this  study: 

•  In  the  future,  further  quantitative  studies  may  be  useful  to  test  trends  noted  in  the 
present  study  on  satisfaction  with  and  use  of  teleradiology.  As  in  the  ENT  study, 
telephone  interviews  or  focus  groups  would  most  likely  obtain  more  comments  than 
in  self-administered  surveys. 

•  Data  on  actual  patient  outcomes  and  telemedicine  use  over  time  could  be  correlated 
with  provider  satisfaction  data  to  obtain  a  more  complete  picture  of  telemedicine  use. 
Although  the  present  study  included  21%  of  Navy  radiologists,  acquiring  a  larger, 
stratified  sample  —  including  multiple  sites  where  teleradiology  is  being  used  — 
would  provide  results  that  would  not  be  specific  to  one  location.  Conversely,  studies 
centering  on  the  use  of  teleradiology  in  a  specific  type  of  setting  could  reveal  details 
that  will  assist  in  developing  teleradiology  for  that  specific  setting.  For  example,  what 
is  the  impact  of  teleradiology,  if  any,  on  emergency  care? 

•  Assessing  the  teleradiology  satisfaction  of  remote  treatment  site  providers  and 
shipboard  providers  would  provide  a  different  perspective  on  teleradiology  using  the 
same  satisfaction  tool. 

•  Studies  of  teleradiology  on  a  larger  scale  could  be  beneficial  for 
large-scale  planning.  For  example,  such  studies  could  aid  the  application  of 
telemedicine  to  casualty  receiving  ships,  where  the  majority  of  the  casualties  would 
be  seen  in  a  wartime  setting,  or  in  a  humanitarian  setting  for  peacetime  missions. 
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The  focus  of  this  study  was  to  assess  levels  of  satisfaction  and  experience  among  radiologists  using  teleradiology  and  other 
telemedicine  technologies  at  the  Naval  Medical  Center,  San  Diego  (NMCSD).  This  study  was  conducted  using  a  survey  administered 
to  resident  radiologists,  radiologists,  and  radiology  specialists.  The  survey  information  was  used  to  compare  and  contrast  the  attitudes 
of  users  and  measure  their  recent  use  of  telemedicine.  This  study  included  assessment  and  comparison  of  radiology-specific 
technologies  not  addressed  in  previous  studies,  such  as  wireless  radiography  and  computed  radiography  (CR).  A  secondary  focus  of 
this  study  was  to  assess  the  generalizability  of  the  satisfaction  survey  instrument  used  across  other  telemedicine  disciplines. 

A  60-item  survey  assessing  satisfaction  with  telemedicine  technologies  was  administered  to  20  radiologists  at  NMCSD.  Data  from 
a  previous  ear/nose/throat  (ENT)  specialist  is  included  as  a  comparison  with  data  collected  in  the  present  study.  Questions  assessing 
general  perceptions  of  telemedicine  were  the  same  for  both  studies.  The  differences  between  the  two  surveys  were  primarily  in  details 
of  technologies  listed  as  being  specific  to  each  specialty.  Items  measuring  satisfaction,  perception,  and  usefulness,  as  well  as  history 
of  use  and  demographic  data  were  presented.  A  final  open-ended  question  was  included  to  capture  other  possible  answers  not 
included  in  the  scales. 

Respondents,  ENT  and  radiology  personnel  alike,  were  generally  satisfied  with  the  telemedicine  technology  and  process  and  had 
very  similar  positive  attitudes  about  telemedicine  in  general.  The  ENT  medical  personnel  responded  positively  to  specific 
telemedicine  technologies  specific  to  their  practice,  (e.g.,  Referral  Management  System  and  video  teleconferencing),  and  the 
radiology  personnel  responded  positively  to  specific  radiological  technologies  (e.g.,  wireless  and  CR). 

The  success  in  using  the  satisfaction  survey  instrument  across  these  two  telemedicine  disciplines  indicates  that  it  may  be 
generalizable  across  all  medical  specialties. 
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